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Abstract Automated ballistics imaging and analysis systems, such as the integrated ballistic identi- 
fication system (IBIS), have advanced gun law enforcement operations by allowing images of gun crime 
evidence to be rapidly compared to a large inventory of evidence collected from other crime scenes. After 
firearm examiners confirm candidate matches, detectives can use the information generated by the links 
among gun crimes to help solve their cases. Links between recovered cartridge casings represent the vast 
majority of ballistic matches made through IBIS. Unfortunately, the two-dimensional grayscale digital 
photography used by the original IBIS equipment was often not refined enough to suggest potential 
matches between often highly damaged bullets from separate crime scenes. Through the acquisition of 
three-dimensional measurements, the BulletTrax-3-D image acquisition technology was specifically de- 
signed to improve the ability of law enforcement agencies to make bullet matches through automated 
ballistic imaging and analysis. This study examines the impact of 3-D imaging technology on the produc- 
tivity of the Boston Police Department’s Ballistics Unit in making bullet matches. When directly compared 
with 2-D imaging technology, we find that the 3-D imaging technology acquired larger amounts of crime 
bullet evidence and was associated with a nearly fivefold increase in the cold bullet matches by the Ballistics 
Unit. Interviews with Boston Police detectives confirmed the considerable investigative value of the cold 


bullet matches. 
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Introduction 


While gun violence has decreased since the early 1990s, 
many Americans still die by gunfire, and many more are 
still affected by non-fatal gun violence. In 2009, a total of 
9,146 people were murdered with firearms (U.S. Dept. of 
Justice—Federal Bureau of Investigation 2009), and it is es- 
timated that another 48,158 were treated in hospitals for 
gunshot wounds received in assaults (Centers for Disease 
Control and Prevention 2010). Research suggests that the 
bulk of urban gun violence is committed by a relatively 
small number of highly active criminals (e.g., see Braga 
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et al. 2001). Crime guns are durable goods and, if not 
recovered by law enforcement, can be used in multiple 
violent incidents. This presents a challenge and an oppor- 
tunity for strategic gun law enforcement operations. 
When fired or ejected from firearms, bullets and car- 
tridge casings pick up signature markings assumed to be 
unique to the gun (Bureau of Alcohol, Tobacco, Firearms, 
and Explosives [ATF] 2001; National Research Council 
2008). Impact with the firing pin, breech face, and ejec- 
tor cause signature marks on the cartridge casing, while 
the gun barrel leaves striations on the bullet as it passes 
through. Firearm examiners can link a gun to crimes by 
analyzing these signature markings. Automated ballistics 
imaging and analysis systems, such as Forensic Technol- 
ogy Inc.’s Integrated Ballistic Identification System (IBIS), 
have advanced gun law enforcement operations by allow- 
ing images of gun crime evidence to be rapidly compared 
to a large inventory of evidence collected from other crime 
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scenes (ATF 2001; Braga and Pierce 2004). After firearm 
examiners confirm candidate matches, detectives can use 
the information generated by the links among gun crimes 
to move their investigations forward and, hopefully, hold 
offenders accountable for their crimes. 

Links between cartridge casings represent the vast ma- 
jority of ballistic matches made through IBIS (Inspec- 
tor General 2005). Unfortunately, the two-dimensional 
grayscale digital photography used by the original IBIS 
is often not refined enough to suggest potential matches 
between crime bullets on a routine basis (National Re- 
search Council 2008). This is very problematic for gun 
law enforcement operations as investigative leads that 
could be generated from existing evidence are not being 
made. Forensic Technology Inc. has continued to improve 
the performance of its automated ballistic imaging and 
analysis systems and, in 2006, started to release a new 
family of IBIS products based on the acquisition of three- 
dimensional measurements from bullets and cartridge 
casings. The BulletTrax-3-D image acquisition technology 
was specifically designed to improve the ability of law 
enforcement agencies to make ballistic bullet matches 
through automated ballistic imaging and analysis. 

In this paper, we assess the performance of the 
BulletTrax-3-D system in assisting law enforcement per- 
sonnel in making “cold” ballistic matches, commonly 
called “hits” by investigators, of crime bullets where link- 
ages are made between cases where previously there had 
been no known connection between the investigations. 
This study does not attempt to address the accuracy of 
computerized matching of 3-D ballistic bullet images, the 
value added by expanding ballistic imaging to include 
non-crime guns, or any other issues in the larger national 
debate on ballistics imaging technology. Rather, this re- 
search examines the impact of new 3-D ballistics imaging 
technology on the productivity of the Boston Police De- 
partment’s (BPD) Ballistics Unit in making cold ballistic 
bullet hits across crime scenes. 


The Emergence of the IBIS System and the Limits of 
2-D Technology in Ballistic Imaging 


In the past, the comparison of ballistics evidence was 
a very labor-intensive and time-consuming task as each 
piece of newly recovered evidence had to be manually 
compared to a potentially vast inventory of recovered or 
test-fired bullets and cartridge casings (ATF 2001). There 
was no means to automate the process and, given labor 
and time constraints, it was very difficult to make matches 
across crime scenes (ATF 2001). During the early 1990s, 
Forensic Technology Inc. developed the IBIS automated 
ballistics imaging and analysis system. The IBIS system 
maintains a computerized database of digital ballistic im- 
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ages of bullets and casings from crime guns. As new im- 
ages are entered, IBIS compares the recovered evidence to 
existing images from prior crime scenes to identify pos- 
sible matches. For cartridge casings, digital images are 
correlated based on characteristics such as breech face, 
firing pin, and ejector marks. For bullets, digital images 
of the lands (grooves from the rifling of the firearms 
barrel) are correlated. The IBIS technology quickly sorts 
through large volumes of ballistics evidence and suggests 
a small number of candidate cases that may match the 
evidence in question. Firearms examiners then manually 
look at the candidate matches and conduct a standard 
forensic comparison to confirm a match, if one actually 
exists. 

In 1999, the Bureau of Alcohol, Tobacco, Firearms, and 
Explosives (ATF) and the Federal Bureau of Investigation 
(FBI) selected IBIS as the standard ballistics imaging tech- 
nology for their then-proposed National Integrated Ballis- 
tics Information Network (NIBIN)(ATF 2001). Implemented 
in 2000, NIBIN seeks to unify and coordinate federal, state, 
and local law enforcement efforts to use ballistics technol- 
ogy to analyze and match recovered gun crime evidence 
within a jurisdiction and across different jurisdictions. 
NIBIN is intended to allow law enforcement to check digi- 
tal images of recovered ballistic evidence against a nation- 
wide file of many thousands of images of ballistics crime 
scene evidence. ATF has made a concerted effort to pub- 
licize the potential value of the technology in advancing 
its mission of apprehending violent gun criminals. As of 
March 2011, the NIBIN initiative had 155 participating 
agencies and 235 IBIS systems; since the program started, 
over 2 million pieces of crime scene evidence have been 
acquired and over 40,000 cold ballistic hits have been 
logged (ATF 2011). 

As the IBIS technology developed, a number of firearms 
examiners and other researchers tested the system’s per- 
formance. Some analysts scrutinized particular parts of 
the IBIS comparison process, such as the default compari- 
son pass and 20 percent threshold for detailed image scor- 
ing and ranking (e.g., George 2004a, b). However, a bulk 
of the research focused on specific performance issues 
raised by the creation of large-scale reference ballistic im- 
age databases (e.g., Tulleners 2001; Thompson et al. 2002; 
DeKinder 2002). This research agenda was influenced by 
the passage of laws in Maryland and New York requir- 
ing ballistics “fingerprinting” of all handguns in their re- 
spective states and the calls of gun control advocates for 
a national reference database of ballistic images for the 
more than 250 million firearms in the United States (Brady 
2002). Gun rights advocates countered that ballistic image 
databases were unreliable, expensive, and had little util- 
ity in solving crimes (Kopel and Burnett 2003). These crit- 
ics also suggested that ballistics imaging technology was 
limited because determined criminals can alter markings 
on or replace barrels, slides, extractors, and firing pins. 
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These modifications would alter the telltale markings 
on ballistics evidence and prevent matches from being 
made. 

Based on their review of the available scientific evi- 
dence, a panel of experts convened by the National Re- 
search Council (2008) concluded that a national reference 
ballistic image database of all new and imported guns 
was not advisable at that point in time. The panel ob- 
served, among other concerns, that the two-dimensional 
(2-D) grayscale images were useful for gross categorization 
and sorting of large quantities of evidence that represent 
routine law enforcement applications in a relatively mod- 
est pool of crime scene and test-fire images. However, the 
technology was much less reliable in distinguishing the 
extremely fine individual markings that would be neces- 
sary to make successful matches in enormous reference 
databases that would contain very large numbers of im- 
ages with very similar gross class and subclass character- 
istics. 

The National Research Council (2008) panel also con- 
cluded that the NIBIN program provided valuable crime- 
solving assistance to law enforcement agencies and that 
the program should be made more effective through tech- 
nological and operational improvements. Among other 
technological recommendations, the National Research 
Council (2008) panel observed the potential promise 
of changing the ballistic imaging standard from 2-D 
grayscale photography to three-dimensional (3-D) surface 
measurement using noncontact microscopy. Briefly, 3-D 
imaging technology captures three-dimensional data on 
ridges and valleys from the striations on the surface of a 
fired bullet to create a digital map of its surface topogra- 
phy that can be mathematically processed. Experiments 
conducted for the panel by the National Institutes of Stan- 
dards and Technology concluded that the 3-D imaging 
technology was superior in matching certain cartridge 
markings when compared to the existing 2-D technology 
(Vorburger et al. 2007). The panel cautiously concluded 
that further testing and development was necessary be- 
fore full consideration could be given to adopting 3-D as 
the new standard in ballistic imaging. 

The National Research Council (2008) report also sug- 
gested that the advancement of 3-D ballistic imaging tech- 
nology could be helpful in improving match rates for 
crime bullets. The vast majority of NIBIN cold hits linked 
cartridge casings rather than bullets. Recovered bullets 
are often mushroomed and fragmented due to hitting in- 
tended and unintended targets. The IBIS 2-D technology is 
not very effective in detecting potentially identical mark- 
ings among bullet images when compared to detecting 
potentially identical markings among less damaged and 
distorted cartridge casings recovered from crime scenes. 
The U.S. Department of Justice (DOJ) Inspector General’s 
audit (2005) found that many NIBIN agencies were not 
entering bullet evidence into the system because firearms 
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examiners believed that the IBIS equipment did not pro- 
duce good quality images of the bullets. 

A recent study found that 3-D ballistic imaging tech- 
nology produced improved match correlations when di- 
rectly compared to match correlations produced by 2-D 
technology. Beauchamp and Roberge (unpublished data) 
analyzed samples of bullets of known matched pairs of 
9-mm Luger, .45 Auto, .38, and .22 Long Rifle calibers, 
and of two material compositions, copper jacketed and 
lead. They found that the 3-D data correlation yielded a 
10 to 15 percent improvement over the 2-D data correla- 
tion. Interestingly, they also found that the correlation 
of the 3-D data combined with the 2-D data yielded a 25 
percent improvement over the 2-D data correlation alone. 
Beauchamp and Roberge (unpublished data) concluded 
that 3-D data and 2-D data provided complementary in- 
formation in the sense that a match not found by one 
algorithm may occasionally be found by the other. The 
3-D matching performance was also noted to be better for 
copper jacketed bullets than for lead bullets. 

Most of the National Research Council (2008) panel’s 
operational recommendations involved practical steps to 
enhance the capacity of law enforcement agencies to col- 
lect, enter, and analyze gun crime evidence using the IBIS 
technology. Law enforcement agencies differ in the de- 
gree to which they use NIBIN resources and, as a result, 
differ markedly in the benefits they derive in establish- 
ing links between gun crimes and generating investiga- 
tive leads. The DOJ Inspector General’s audit (2005) found 
that, as more evidence was entered into NIBIN, law en- 
forcement agencies greatly improved the likelihood that 
ballistic matches were identified. The DOJ audit reported 
that the 20 percent of NIBIN partner agencies that con- 
tributed 75 percent of the evidence to the NIBIN system 
also identified 72 percent of the cold hits. 

Particular law enforcement agencies, such as the 
Boston Police Department (BPD), report considerable 
value added to their gun law enforcement operations by 
adopting and diligently using the IBIS ballistic imaging 
technology (Braga and Pierce 2004; Braga 2008). Since 
adopting the 2-D technology in March 1995, the BPD 
makes an aggressive effort to collect, image, and enter bal- 
listic evidence from all incidents involving firearms, rang- 
ing from homicides to illegal possession cases to non-fatal 
gun assaults, into the IBIS database. The DOJ Inspector 
General (2005) audit report included the BPD as one of 12 
agencies selected for a closer examination of hit rates. At 
the time the data were collected by DOJ, the BPD reported 
the highest number of total cartridge cases entered into 
the IBIS system (5,741) and the second highest number of 
total cold hits reported (431). 

A recent assessment of the impact of ballistics imaging 
technology in Boston found that the adoption of IBIS was 
associated with a six-fold increase in the monthly number 
of cold hits made by the BPD Ballistics Unit (Braga and 
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Pierce 2004). Extended analyses of the cold hits reported 
that almost 39 percent generated significant investigative 
leads; IBIS cold hits were also associated with the arrest 
of very serious gun offenders and high conviction rates 
for their immediate gun offenses (Braga 2008). Cartridge 
casing matches generated most of these benefits, however. 
Bullet matches represented only 2 percent of the cold hits 
(8 of 396) made by the BPD during the 1995-2002 study 
period (Braga and Pierce 2004). 


The Adoption of the BulletTrax-3-D Technology by 
the Boston Police Department 


The BPD was selected as the site for an evaluation of the 
adoption of the BulletTrax-3-D technology based on the 
agency’s reputation as engaging innovative gun violence 
reduction strategies and their commitment to the bal- 
listic imaging of recovered gun crime evidence (e.g., see 
Braga et al. 2001; Braga and Pierce 2004). The BPD and FTI 
signed a memorandum of understanding to cooperate in 
an evaluative study to assess the value added by 3-D bullet 
imaging technology to their gun law enforcement oper- 
ations in August 2008. Over the course of fall and early 
winter 2008, BPD acquired and tested the IBIS BulletTrax- 
3-D bullet imaging system and IBIS MatchPoint+, a com- 
parative analysis system for 2-D and 3-D images. The 3-D 
system was fully operational as of January 1, 2009. The 
evaluation study period began at this time and continued 
until June 30, 2010. 

Figure 1 presents the yearly counts of bullets and yearly 
counts of cartridge casings entered into ballistic imag- 
ing systems by the BPD Ballistics Unit between 1995 and 
2010. Excluding the 1996, 1997, 2007, and 2008 years, the 
Ballistics Unit entered an average of 332 bullet images 
each year. According to Steven Faulkner, a long-time BPD 
IBIS technology operator, the low bullet image acquisition 
counts during those years was due to the directives of unit 
supervisors who believed that scarce resources should not 
be used to image all recovered crime bullet evidence. This 
directive was based on the supervisors’ observations that 
the 2-D imaging technology was not very useful in gener- 
ating cold bullet hits. After adopting BulletTrax-3-D, the 
BPD Ballistics Unit made 366 bullet entries in 2009 and 
314 entries in 2010. 

The yearly counts of cartridge casings entered into 
the IBIS 2-D imaging system (Figure 1) generally tracked 
yearly counts of gun homicide and gun assault incidents 
between 1995 and 2008 (Figure 2). Cartridge casing en- 
tries dropped off significantly during 2009 and 2010. 
According to Captain Detective Michael Broderick, then 
commander of all forensics operations for the BPD, the de- 
crease in cartridge casing entries during these years was 
linked to the Ballistic Unit’s participation in the evalu- 
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ation study. As will be discussed further below, the im- 
pact evaluation required all bullet evidence to be imaged 
twice: once in the 2-D system and once in the 3-D sys- 
tem. This “double entry” procedure was necessary to make 
valid comparison in the performance of the two imaging 
systems. However, given limited resources, the Ballistic 
Unit’s ability to keep up with cartridge casing acquisi- 
tions was diminished during the 18-month study time 
period. This should be considered an artificial condition 
necessary to complete a rigorous evaluation and not an 
indication that bullet entries somehow undermine car- 
tridge casing entries. 

Figure 3 presents the yearly number of cold ballistic 
bullet matches made by the BPD Ballistics Unit between 
March 1995, when the IBIS 2-D technology was first im- 
plemented in Boston, and December 2010. The Ballistics 
Unit averaged one bullet match per year between 1995 
and 2003 and made zero bullet matches per year between 
2004 and 2008. For the entire pre-test time period, the 
yearly average number of cold bullet hits was 0.64. The 
Ballistics Unit then adopted the BulletTrax-3-D technol- 
ogy and, in 2009 and 2010, averaged 4.5 bullet matches 
per year. All four hits reported in 2010 were made during 
the evaluation study period. 


Impact Evaluation 


To measure the impact of the BulletTrax-3-D technology 
on the productivity of the BPD Ballistics Unit, it is im- 
portant to consider the nature of the technological inno- 
vation and its potential impact on BPD operations. The 
BulletTrax-3-D technology can cross-examine large vol- 
umes of crime bullet evidence and suggest asmall number 
of candidate cases that may match the evidence in ques- 
tion. The firearms examiner then carefully looks at the 
candidate cases using standard procedures to determine 
whether a real match actually exists. The nature of a con- 
firmed match and its utility to a criminal investigation 
does not change as a result of the 3-D technology. Foren- 
sic evidence, such as ballistics matches, is one part of an 
information chain (such as eyewitness testimony, circum- 
stantial evidence, etc.) that leads to the ultimate arrest 
and prosecution of gun criminals. Arrest and prosecution 
is influenced by many factors beyond the forensic link of 
a particular gun to ballistics bullet evidence collected at 
separate crime scenes. 

Since the value of a ballistics bullet match to the res- 
olution of a crime is not meaningfully different before 
and after the adoption of 3-D technology, the outcome 
of interest is the number of bullet matches made by the 
Ballistics Unit, not the ultimate disposition of the result- 
ing cases. While we do assess the investigative value of 
the bullet matches in a subsequent section, the central 
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question considered by this study is whether the 3-D tech- 
nology changes the ability of the BPD Ballistics Unit to 
link gun crimes by making crime bullet and bullet frag- 
ment matches. Stata /IC 11.0 statistical software was used 
to analyze the data presented below. 


Bullet Trax-3-D Imaging System v. Heritage 2-D 
Imaging System Comparison 


During the January 1, 2009, through June 30, 2010, study 
time period, the BPD Ballistics Unit attempted to image 
and enter all recovered crime bullet evidence and bullets 
that were test-fired from recovered crime guns into both 
the BulletTrax-3-D system and the heritage IBIS 2-D sys- 
tem. The attempted double entry of all acquired crime 
gun evidence represents the most rigorous test of the abil- 
ity of the new 3-D imaging technology to make cold hit 
bullet matches relative to the 2-D technology. Both tech- 
nologies were used to analyze the same evidence, operated 
by the same support staff, and candidate bullet matches 
were analyzed under the microscope by the same firearm 
examiners. 

The Ballistics Unit attempted to enter 650 pieces of 
crime bullet evidence into both systems over the course of 
the study. During the pilot study period, BPD BulletTrax- 
3-D and heritage 2-D system operators were instructed to 
spend up to two hours of effort on the image acquisition of 
each individual recovered crime bullet. Unfortunately, 63 
recovered crime bullets (9.7% of 650) could not be entered 
into either the 3-D system or the 2-D system. The reasons 
that these crime bullets could not be entered included: 
44.4 percent were .22 copper wash bullets that could not 
be acquired due to “slippage” (28 of 63), 34.9 percent were 
lead bullets that were too damaged (22 of 63), and 20.7 
percent were bullets with poor markings that were fired 
from worn barrels or altered barrels that produced stria- 
tions that ran together (13 of 68). The BulletTrax-3-D sys- 
tem acquired 587 crime bullet images, while the heritage 
2-D system acquired only 520 crime bullet images. This 
represents a 12.9 percent improvement in the ability of 
the Ballistic Unit to image and analyze crime bullet evi- 
dence. The most common reasons for the differential en- 
tries were that the crime bullets were too damaged (40.3%, 
27 of 67) or had poor markings because they were fired 
from worn or altered barrels (26.9%, 18 of 67). 

Using the BulletTrax-3-D imaging technology, the Bal- 
listics Unit made nine cold bullet hits from the acquired 
587 crime bullet images. This represents a 1.53% hit rate 
(9/587*100). The Ballistic Unit did not make any cold 
bullet hits based on the 520 crime bullet images acquired 
by the heritage 2-D imaging technology (or a 0.0% hit rate, 
0/520). We tested a null hypothesis that there is no differ- 
ence in the proportion of cold bullet hits made by the 3-D 
system as compared to the proportion of cold bullet hits 
made by the heritage 2-D system. A simple difference of 
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two sample proportions’ z-score test was used to test the 
null hypothesis. Based on the size of the z-score that was 
generated by the test (z = 2.83) and associated p-value (p < 
0.0023), we rejected the null hypothesis and concluded 
that the difference between these two proportions was 
so large that it was very unlikely to have happened by 
chance alone. This simple comparison suggested that the 
3-D system generated a significantly higher proportion of 
cold bullet hits when compared to the proportion of cold 
bullet hits generated by the heritage 2-D system under the 
same conditions. 

We also conducted a simple comparison of the cold bul- 
let hit rates made by the Ballistics Unit over time. Between 
March 1, 1995, and December 31, 2008, the Ballistics Unit 
imaged 3,327 crime-related bullets in their heritage IBIS 
2-D imaging system. During this 166-month pre-test pe- 
riod, only nine cold bullet hits were made by the heritage 
2-D imaging system. The proportion of bullet evidence 
entries that resulted in a cold hit is 0.27% (9/3,327*100). 
Using a simple difference of two sample proportions’ z- 
score test, we concluded that there was a statistically sig- 
nificant increase in the cold bullet hit rate when the Bal- 
listics Unit adopted the BulletTrax-3-D technology (z = 
4.16, p < 0.000). If the 3-D technology was available dur- 
ing the pre-test time period and the Ballistics Unit made 
matches at the same rate as during the post-test period, 
they would have made approximately 51 cold bullet hits 
during the pre-test time period (1.53% rate*3,327 = 50.9). 
This represents roughly 42 additional cold bullet hits or 
an estimated 467% increase over the performance of the 
heritage 2-D system. 

In addition to the nine cold hits made during the 18- 
month pilot project time period, the Ballistics Unit also 
made eight “warm” hit matches using the BulletTrax-3-D 
technology. Warm hit ballistic matches occur when in- 
vestigators have a priori knowledge that multiple gun 
crimes, or recovered firearms and incidents of gun vio- 
lence, might be linked. In these eight warm hits, the sug- 
gested matches were initially examined via BulletTrax-3-D 
and, when potential matches were identified by the tech- 
nology, a firearms examiner then confirmed the candi- 
dates using a microscope. It is important to note here that 
the BulletTrax-3-D equipment operators and the firearms 
examiners were blind to whether the matched crime bul- 
lets from separate cases were previously linked by ex- 
isting investigative knowledge held by BPD detectives. 
The determination of warm and cold hits occurred af- 
ter the match was confirmed and the information was 
forwarded to the appropriate case detectives for their 
consideration. 

While warm hits do not generate previously unknown 
investigative links, the confirmed connections are still 
important to investigators as they develop cases against 
gun offenders. For instance, the BulletTrax-3-D system 
confirmed that the 9-mm handgun recovered from a 
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hollowed-out copy of Gray’s Anatomy in the apartment 
of Philip Markoff, the so-called Craigslist killer, was in- 
deed the same firearm that was used to kill 25-year-old 
Julissa Brisman in a Boston hotel. While there was plenty 
of other evidence to arrest Markoff, confirmation that 
Boston Police homicide detectives had recovered the mur- 
der weapon was an important investigative development. 
Summing the warm and cold hits, the total hit rate for 
the BulletTrax-3-D technology during the pilot period was 
2.9% (17/587*100). 


Multivariate Pre-Test v. Post-Test Comparison of the 
Productivity of the BPD Ballistics Unit 


Our deeper analysis of the impact on the monthly num- 
ber of cold bullet hits made by the BPD Ballistics Unit 
associated with the adoption of the BulletTrax-3-D tech- 
nology follows a basic one-group interrupted time series 
design (Campbell and Stanley 1966; Cook and Campbell 
1979). Unfortunately, we can only directly compare the 
performance of the BulletTrax-3-D technology to the her- 
itage 2-D technology for the 18-month pilot project time 
period. We can, however, examine the impact of the tech- 
nological change on the productivity of the Ballistics Unit 
over the entire 184-month time period between March 
1995 and June 2010. 

In absence of a separate comparison group, the key to 
a compelling one-group interrupted time series design is 
the degree to which forces not related to the interven- 
tion influence the outcome variable. If at the time the 
BulletTrax-3-D technology was introduced in Boston the 
staff of the Ballistics Unit also changed, this could pose 
a problem of a change in “instrumentation” (Cook and 
Campbell 1979, Shadish, Cook, and Campbell 2002). How- 
ever, this was not the case in Boston. The unit had, on 
average, about five firearm examiners and 10 total per- 
sonnel on staff between 1995 and 2008 (Table 1). The total 
personnel dropped by only one staff member between 
2009 and 2010 as the unit lost two firearm examiners and 
hired an additional support staff member. The only sub- 
stantive change in the dynamics of the Ballistics Unit was 
the addition of the BulletTrax-3-D technology. 

Factors external to the Ballistics Unit that might poten- 
tially influence bullet matches were controlled for in our 
design though a time series analysis of the monthly counts 
of bullet matches before and after the introduction of 
BulletTrax-3-D technology. Monthly counts of cold bullet 
hits are distributed in the form of rare event counts. There 
are well-documented problems associated with treating 
event count variables, which are discrete, as continuous 
realizations of a normal data generating process (King 
1989). As such, methods such as standard mean difference 
tests and ordinary least squares regression that assume 
population normality of the dependent variable should 
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Table I. Staffing Levels of the BPD Ballistics Unit, 1995 — 
2010 


Year Supervisors Firearm Examiners Support Staff Total 


1995 1 7 1 9 
1996 2 7 1 10 
1997 2 6 2 10 
1998 2 5 3 10 
1999 2 6 3 11 
2000 2 5 3 10 
2001 2 5 3 10 
2002 1 4 4 9 
2003 1 5 5 11 
2004 2 4 4 10 
2005 1 4 4 9 
2006 1 5 4 10 
2007 1 5 4 10 
2008 2 5 3 10 
2009 1 5 4 10 
2010 1 3 5 9 


not be used to analyze count data (Gardner, Mulvey, and 
Shaw 1995). Rather, Poisson regression is generally used to 
estimate models of the event counts (Long 1997). The Pois- 
son regression model has the defining characteristic that 
the conditional mean of the outcome is equal to the condi- 
tional variance. The outcome variable in this analysis, the 
monthly count of cold bullet matches, has a mean = 0.10 
and variance = 0.11. A Komologorov-Smirnov goodness- 
of-fit test of the outcome variable distribution strongly 
indicated that it was generated by a Poisson process (KS = 
0.087, p < 0.999). 

The pre-BulletTrax-3-D time series was comprised of 
the monthly counts of cold bullet hits between March 
1995 and December 2008; the post-BulletTrax-3-D time se- 
ries was comprised of monthly counts of cold bullet hits 
between January 2009 and June 2010. A binary dummy 
variable indicating whether the BulletTrax 3-D technol- 
ogy was present or not was constructed to estimate the 
effects of the intervention on the monthly counts of cold 
bullet hits. It is important to recognize that the monthly 
number of cold bullet hits made by the BPD Ballistics Unit 
is influenced by the amount of evidence acquired by the 
unit every month. We included the monthly number of 
bullet images entered in our statistical models to control 
for the amount of evidence being entered into the ballis- 
tic imaging systems over time that may or may not lead 
to a bullet match. We also included the monthly number 
of cartridge casings entered into the IBIS ballistic imag- 
ing system in our statistical models as another measure 
of the workload experienced by the Ballistic Unit during 
the study time period. In order to account for seasonal 
variations in our model, we included dummy variables 
for each month. Finally, to control for the effects of as- 
saultive gun violence levels on broader BPD enforcement 
operations, we included the aggregate monthly counts 
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of firearm homicide incidents and firearm aggravated as- 
sault incidents during the study time period. 

The parameters for the independent variables were ex- 
pressed as incidence rate ratios (1.e., exponentiated co- 
efficients). Incidence rate ratios are interpreted as the 
rate at which things occur; for example, an incidence 
rate ratio of 0.90 would suggest that, controlling for 
other independent variables, the selected independent 
variable was associated with a 10% decrease in the rate 
at which the dependent variable occurs. To ensure that 
the coefficient variances were robust to violations of the 
homoskedastic errors assumption of linear regression 
models, Huber/White/sandwich robust variance estima- 
tors were used. Following social science convention, the 
two-tailed, .05 level of significance was selected as the 
benchmark to reject the null hypothesis of “no differ- 
ence.” The basic model was as follows: 


Monthly Count = Month + Monthly Count of Bullet 
Entries + Monthly Count of Cartridge Entries 
+ Monthly Count of Bullet Matches Dummies of 


Gun Crimes + BulletTrax Dummy + Error 


Table 2 presents the results of the Poisson regression 
model. The monthly count of bullet and cartridge casing 
evidence being imaged and entered by the Ballistics Unit 
did not have a statistically significant relationship to the 
monthly counts of cold bullet matches over time when 
the other covariates were controlled in the model. How- 
ever, controlling for the other covariates, an increase in 
the monthly counts of gun crimes in Boston was associ- 
ated with a decrease in the number of monthly bullet 


Table 2. Results of Poisson Regression Model (N = 184) 


Robust 
Variable IRR. Std. Error Z P> |Z| 
BulletTrax-3-D 4.694 2.461 2:95 0.003 
Bullets entered 1.017 0.011 1.54 0.124 
Casings entered 0.989 0.009 —1.18 0.239 
Gun violence 0.939 0.025 —2.29 0.022 
February 1.555 1.177 0.58 0.559 
March 1.099 0.981 0.11 0.915 
April 0.336 0.430 —0.85 0.394 
May 0.509 0.656 —0.52 0.601 
June .077 0.877 0.09 0.927 
July 1.128 1.055 0.13 0.897 
August 0.001 0.008 —20.59 0.000 
September 2.141 1.576 1.03 0.301 
October 0.009 0.007 —21.00 0.000 
November 0.363 0.305 —1.20 0.229 


December 0.009 0.007 —21.39 0.000 


Pseudo R? = 0.3397 

Wald chi-square = 3063.54; degrees of freedom = 15; p < 0.000 
Note: January was the reference category for the month dummy 
variables. 
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hits made by the Ballistics Unit. Certain month dummy 
variables were also statistically significant, suggesting sea- 
sonal differences in cold bullet hits generated by the Bal- 
listics Unit. Turning to the potential impact of BulletTrax- 
3-D, we find the adoption of the technology, controlling 
for the other independent variables, was associated with a 
statistically significant 4.69-fold increase in the monthly 
number of cold bullet hits generated by the BPD Ballistics 
Unit (p < .01). 


Investigative Value of Cold Bullet Hits 


As described earlier, BulletTrax-3-D cold bullet hits do 
not change the inherent value of ballistics evidence in 
an investigation, as many other factors converge to deter- 
mine whether an arrest is eventually made. A cold hit that 
links gun crimes, whether it involves bullets or cartridge 
casings, does not guarantee an immediate arrest. For in- 
stance, the linked incidents may not involve victims or 
witnesses that can or are willing to make a positive identi- 
fication. Rather, cold hits better position law enforcement 
agents to identify gun criminals through strategic anal- 
yses of information and intelligence sharing. The Boston 
Police and their criminal justice partners use cold hit in- 
formation as a tool to shed light on the dynamics at play in 
violent street social networks and to further their ability 
to apprehend violent gun criminals (Braga 2008). IBIS 2-D 
and BulletTrax-3-D systems provide Boston investigators 
with more opportunities to make links among individu- 
als and locations through a particular gun. 

We interviewed Boston Police detectives that were as- 
signed to investigate the linked gun crimes for the nine 
cold bullet hits made by the BulletTrax-3-D technology 
to determine the investigative value of each hit. Table 
3 presents the types of gun crimes that were linked in 
each cold hit, a brief synopsis of the gun crimes that were 
linked, and the interviewed detectives’ assessment of the 
investigative value of the hits. These nine cold bullet hits 
linked three homicide incidents (including a triple homi- 
cide), seven gun assault incidents, four illegal gun pos- 
session incidents, and one found gun incident. All nine 
cold bullet hits were regarded as noteworthy investigative 
leads by the interviewed detectives, and the significance 
of each lead was determined by the investigative and intel- 
ligence information available from the linked gun crimes. 

The detectives reported that two cold hits were di- 
rectly associated with new arrests or the issuance of ar- 
rest warrants; in short, it would not have been possible to 
hold these offenders accountable for their violent crimes 
without the information linked by the cold bullet hit. 
For instance, in cold bullet hit number 4, a Taurus .40 
Model PT 140 Pro semiautomatic pistol recovered from a 
non-fatal gun assault offender was linked to triple homi- 
cide incident that occurred four months earlier. A sub- 
sequent interview with the arrested gun assault offender 
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Table 3. Investigative Value of Cold Bullet Hits 
Cold Bullet 
Hit Number Linked Gun Crimes Gun Crime Synopsis Investigative Value 
1 (1) Assault with deadly The first incident involved the unsolved There was not enough evidence to arrest the 
weapon-firearm non-fatal shooting of four Hendry Street rival Bowdoin Street gangsters for the 
(2) Illegal gun possession gangsters. Eight months later, the Boston quadruple shooting. The cold bullet hit 
Police chased three Bowdoin Street was considered value investigative 
gangsters were ina stolen motor vehicle.A information as it confirmed intelligence 
Norinco .45 Model 1911A1 semiautomatic that the quadruple shooting was part on 
pistol was recovered from one of the an ongoing dispute among these gangs. 
gangsters when Boston Police The arrested three suspects are in the 
apprehended them. process of being prosecuted for their 
immediate gun crime. 
2 (1) Assault with deadly The first incident involved the non-fatal The cold bullet hit confirmed intelligence 
weapon-firearm shooting of a Cape Verdean Outlaw that the juvenile was the shooter. 
(2) Illegal gun possession gangster. Six days later, BPD Youth However, the Suffolk County District 
Violence Strike Force officers observe a Attorney’s Office declined prosecution of 
juvenile Olney/Norton gangster acting the gun assault incident and pursued the 
suspiciously. The juvenile was arrested illegal gun possession charge. Juvenile was 
after he admitted that he was illegally convicted and served 18 months ina 
carrying a Hi-Point 9-mm Model C-9 secure juvenile facility. 
semiautomatic pistol. 
3 (1) Homicide The first incident involved the homicide ofa _ The cold bullet hit was considered a 
(2) Found gun known street-level drug dealer. Ten days significant lead in the investigation as the 
after the homicide, the Drug Control Unit homicide firearm was recovered. Despite 
and K-9 officers conducted a search ofa the gun recovery, homicide investigators 
nearby empty lot known for illegal drug do not have enough evidence to arrest a 
activity. The officers recovered a Smith & suspect in the case. The investigation is 
Wesson .45 Model 645 semiautomatic ongoing. 
pistol. 
4 (1) Assault with deadly While patrolling a gun violence hot spot, When the homicide detectives interviewed 
weapon-firearm BPD Youth Violence Strike Force witnessed the gun possession suspect, they learned 
(2) Homicide (triple) a non-fatal gang shooting incident. The that their key suspect in the triple 
officers apprehend the shooter and homicide had transferred the gun to the 
recovered a Taurus .40 Model PT 140 Pro gun possession offender. This was viewed 
semiautomatic pistol. The gun was as a very significant lead in the triple 
test-fired and the bullet matched homicide case and, coupled with other 
recovered bullets from a triple homicide investigative information, led directly to 
that occurred nearly four months earlier. the issuance of an arrest warrant for the 
homicide offender. After the recovered 
crime gun was traced, this murder 
investigation resulted in an ATF gun 
trafficking investigation. 
5 (1) Assault with deadly MIC gang member was shot in gymnasium at Boston Police detectives reported that this 
weapon-firearm night during teen dance. About six weeks was a very important lead that confirmed 
(2) Illegal gun possession later, the Drug Control Unit served a a rival gang member perpetrated the 
warrant on the residence of a Greenwood non-fatal shooting. Combined with other 
Street gangster and recovered three guns, evidence (such as phone records), the cold 
cocaine, and money. The guns were bullet hit was associated with the issuance 
test-fired and a bullet fired from one of the of an arrest warrant. 
recovered guns (Ruger .357 Magnum 
revolver) was linked to bullet evidence 
recovered at the earlier non-fatal shooting. 
6&7 (1) Assault with deadly After responding to a “shots fired” call for The arrested suspects were prosecuted for 
weapon-firearm service, Boston police officers arrested two their immediate gun crime. Both were 
(2) Assault with deadly Homes Avenue gangsters for the non-fatal convicted; one received a 5-7 year state 
weapon-firearm shooting a rival Wendover gang member. prison sentence and the other received an 
(3) Assault with deadly The officers recovered a Daewoo .40 Model 8-12 year state prison sentence. The cold 


weapon-firearm 


D40 semiautomatic pistol. The gun was 
test-fired, and the bullet matched crime 
bullets recovered from two prior non-fatal 
shooting incidents. 


bullet hit confirmed prior intelligence 
that the other non-fatal shootings were 
part of an ongoing gang dispute. 


(Continued on next page) 
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Table 3. Investigative Value of Cold Bullet Hits (Continued) 


Cold Bullet 

Hit Number Linked Gun Crimes Gun Crime Synopsis Investigative Value 

8 (1) Homicide The first incident involved the fatal shooting Homicide detectives regarded this cold bullet 

(2) Illegal gun possession of a known street-level drug dealer. Three hit as a valuable lead in the investigation. 

months later, the Drug Control Unit While they acquired the homicide firearm, 
executed a search warrant on the the detectives reported that additional 
apartment of another drug dealer and evidence was necessary to connect a 
recovered crack cocaine, money, and a suspect to the homicide. There is no other 
Hi-Point 9-mm Model C-9 semiautomatic information linking the suspect to the 
pistol. The gun was test-fired and the homicide victim. No arrest warrant was 
bullet matched a crime bullet recovered issued and the investigation is ongoing. 
from the prior homicide. 

9 (1) Shots fired/Illegal gun Boston Police officers arrested a Peacevale Homicide detectives regarded this cold bullet 


possession 
(2) Homicide 


gangster who was shooting at rival gang 
members (no victims found, property 
damage only). A Smith & Wesson 9-mm 
semiautomatic pistol was recovered and 
test-fired. The bullet matched a bullet 


hit as a valuable lead in the investigation. 
Coupled with prior intelligence, the cold 
bullet hit confirmed that two suspects of 
interest committed the homicide. At the 
time of the interview, the investigation 


recovered from a homicide of a street-level was very active and an arrest warrant was 


heroin dealer that occurred one month 


earlier. 


revealed that the lead suspect in the triple homicide in- 
cident had provided Taurus .40 semiautomatic pistol to 
him. Sergeant Detective Marc Sullivan, the lead investiga- 
tor on the triple homicide case, commented, “The bullet 
match was a significant lead in the Boston Police Depart- 
ment’s investigation ofa triple homicide. Through further 
investigation, we were able to link the recovered firearm 
to the homicide offender and obtain an arrest warrant.” 
A gun trafficking investigation also developed from this 
case as a subsequent ATF trace of the recovered firearm 
revealed that a third party had recently purchased the 
Taurus .40 semiautomatic pistol at a licensed dealer in 
an Interstate-95 highway southern state with lax gun con- 
trol laws when compared to Massachusetts tighter gun 
control laws. 


Conclusion 


The results of this research study suggest that the 
BulletTrax-3-D imaging technology significantly im- 
proved the ability of the BPD Ballistic Unit to make cold 
hit bullet matches during the pilot study time period. 
When directly compared with the performance of the 2-D 
imaging system that analyzed the same bullet evidence, 
our impact evaluation found: 


= The BulletTrax-3-D technology was associated with 
a 13% increase in the capacity of the Ballistic Unit 
to image submitted crime bullet evidence. 

= The BulletTrax-3-D made nine cold bullet hits from 
587 crime bullet images acquired during the pilot 
test period (1.53% hit rate), while the heritage 2-D 
system made zero cold hits from 520 crime bullet 


believed to be forthcoming. 


images acquired during the pilot period (0.0% hit 
rate). 

= If the 3-D technology was available during the 
pre-test time period and the Ballistics Unit made 
matches at the same rate as during the post-test 
period, they would have made 42 additional cold 
bullet hits or an estimated 467% increase over the 
performance of the heritage 2-D system. 

=» When examining the March 1995 through June 
2010 time period, multivariate Poisson regression 
models estimate that the BulletTrax-3-D technol- 
ogy was associated with a 4.69-fold increase in the 
ability of the Ballistic Unit to make cold bullet 
hits. 

= The BulletTrax-3-D technology was associated with 
the generation of noteworthy investigative leads in 
the cases linked by the nine cold bullet hits, in- 
cluding the issuance of arrest warrants in two sets 
of linked cases that would not have been possible 
without the cold hit. 


Clearly, the enhanced capacity of the BulletTrax-3- 
D imaging technology represents a significant improve- 
ment over the heritage 2-D system in ballistic imaging 
and analysis. While nine cold bullet hits over an 18-month 
period may seem like a very modest number, it is impor- 
tant to recognize that Boston is a relatively safe city that 
experiences lower levels of gun violence. In 2010, Boston 
had a population of 617,594 residents and experienced 60 
gun homicides and 200 non-fatal shootings (Figure 2). In 
comparison, Baltimore, a city of 620,961 residents, expe- 
rienced 170 gun homicides and 420 non-fatal shootings 
in 2010. Jurisdictions with larger numbers of fatal and 
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non-fatal gun incidents seem likely to generate larger 
numbers of cold bullet hits. 

Of course, it is important to note that the 3-D technol- 
ogy alone is not enough to improve gun law enforcement 
operations. As suggested by the DOJ Inspector General’s 
audit (2005), the technology needs to be accompanied by 
an organizational commitment to image and analyze all 
recovered crime gun evidence in a comprehensive man- 
ner (see also Gagliardi 2010). Law enforcement agencies 
also need to ensure that linked information gets to the 
appropriate criminal investigators and that these inves- 
tigators follow up on all investigative leads. Only then 
will repeat gun criminals face an increased risk of being 
apprehended for their crimes. 
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